Various means of human interaction with computers exist today, but few are entirely suitable for interaction in the first person perspective with computer generated graphical environments. Increasingly, users wish to "travel" within computer generated simulations. (For discussion, games will be considered a subclass of simulations in general here.) While large scale simulators which enclose all or part of an operator's body have existed for some time, they are quite expensive and otherwise impractical for the average user today. In contrast, powerfull personal computers are affordable and are practical, taking up relatively little room and using little electrical power. Such current generation personal computers are capable of running relatively powerful simulations, and a rapidly growing amount of simulation software already exists for them. Perhaps most common are game type simulations, where one or more users interact with an entire simulated environment in tests of mental and dexterous skills. Unfortunately, while computer processing hardware has become quite inexpensive, and while software development capability has become quite powerfull, corresponding change has not occurred for user input/output ("I/O") equipment. This has become a serious bottle-neck for operation of existing generation simulations, as well as for the development of the next generation of simulations.
Prior Art Hardware
Keyboards are one of the oldest, and today remain the most widely used computer input device. Keyboards have a plethora of keys, typically 102, and operate by user pressure to create digital state changes which a computer interprets to represent characters of text.
Unfortunately, for inputting anything other than text, keyboards are very awkward to use. Further, they are generally regarded as incapable of analog type input. (Today little discrete computer equipment uses true analog signals. Instead, high speed digital signals are used to simulate analog. In further discussion herein the term "analog" will generally be used to mean analog appearing, regardless of how achieved or the actual nature of the signals involved.) To smoothly interact with computer graphical environments, analog type input is often preferred or necessary. For these and other reasons, computer users wishing to communicate in real time with computer generated environments overwhelmingly abandon keyboards for other input devices.
The second most widely used input device today is the mouse. Mice provide analog type inputs representing movement in two bounded linear axes (or what some term two degrees of freedom), as well as buttons for providing digital state change inputs. Mice have a number of advantages. They require little training to operate, unlike typing at a keyboard, and users do not need to visually observe the mouse when using it, unlike hunt-and-peck typing. Further, to a limited extent, mice can be used to enter complex variants of linear movement, like velocity and acceleration.
Unfortunately, mice also have disadvantages. The movement range of a mouse is "bounded." It is absolutely bounded by the users reach and often also so by surface dimensions where it is operated (e.g., it can not be moved beyond the boundaries of a mouse pad or the available desktop surface). Further, mice are generally not suitable for inputting orientation data (e.g., pitch, roll, and yaw, or complex variants of these data types).
Somewhat related to mice are trackballs, which are often simplistically portrayed as mice turned upside-down. A key advantage of trackballs is that in addition to linear movement information, they may be used to input orientation and its complex variants. Unlike mice, and joysticks which are discussed below, trackball movement range is not bounded either linearly or orientationally. A trackball can move an infinite "length" or be rotated any possible number of degrees. Unfortunately, trackballs have problems with cross-talk and sometimes produce unnatural movement scenarios (problems discussed more below).
The last of the common individual input devices considered here is the joystick (many other types of input devices exist for specific purposes, but those discussed here are by far the more popular). Stick and rudder controls have existed long before computers. Therefore it was only natural that as computers were used to simulate stick controlled applications a similar input device would be developed. Joysticks have a hand operated lever which can be tilted through two axes (i.e., two degrees of freedom). Almost all joysticks also have additional input devices incorporated into the lever handle, examples of which include digital state changers such as trigger buttons and analog inputs such as thumb pressure pads.
Joysticks require little user training, and unlike all of the previously discussed input devices, they have an inherent center or origin. Further, special features can be built directly into a joystick or provided via software drivers to pre-format the joystick signal content. Examples include degrees of non-linear input scaling and dead-zones where movement produces no input (often implemented as a leaving center dead-zone only; i.e., subsequent handle movements crossing over the zero position do not encounter a "dead" region).
However, even joysticks have limitations, perhaps the most important being that they are orientation bounded. For example, the stick handle portion of a joystick can typically only pivot through a 60 degree conical region.
Still other user input devices are possible (e.g., knobs), but are very little used today, particularly for input to personal computers. Combinations of devices are of course also possible, and in fact almost all PC's today are at some point provided with keyboard and mouse, or keyboard and trackball combinations. When desired, joysticks are usually added to such combinations as an aftermarket accessory. Joysticks are particularly popular with players of computer game simulations, with many such games being inoperable without a joystick for input.
Particular combinations of devices as well as combination devices also deserve note here. Users of computers can only operate one or two input devices concurrently, because they only have two hands, but combination device usage to some extent is possible. Personal computer systems can have both a joystick and a trackball, but due to computer data port constraints, most can only have one such device active at a time. Somewhat of an exception to this is the device called a game pad. Game pads are relatively small lightweight input units designed for the user to pick up and operate in a two-handed manner, using their thumbs for primary data input, but also often having optional input devices like triggers. A typical game pad will have a miniature track ball for operation with one thumb, and a miniature joystick or force sensitive pressure pad for operation with the other thumb. However, the inventor knows of no game pads that "add value" to the basic input device signals from the devices incorporated into a game pad. Finally, an interesting integrated device about to enter the market is a combination joystick-knob unit (the knob is a spinner type device, having no detents or stops). This is a Logitech Corporation product named the WingMan Warrior, which uses the knob to provide yaw control, i.e., a single orientation degree of freedom.
First Person Perspective in Graphical Environments
Simplistically put, users of computer simulations want to do two things. They want to geographically move within simulations and they want to look about within simulations (e.g., within three linear and three orintational dimensions). A third desirable activity within simulations is manipulation, use of the hands within the simulation itself; but that is a complex topic which is not particularly germane here. Unfortunately, in the market today, input devices for personal computers either do not provide ability to accomplish both of these goals with any appreciable independence, or when they do provide independent view control they provide it too independently (i.e., in a manner unlike normal human vision).
Movement is usually the dominant goal in computer simulations. Much like in real environments, users of simulated virtual environments want to advance, retreat, veer to one side or another, and change speed. Since most of these operations are ones where a user needs to look in the direction of movement, or opposite to that direction if movement is backwards, most simulations have simplistically omitted any independent control over where the user looks. The exceptions to this have provided very limited vision control (e.g., field of view control, still centered on the movement axis); or have achieved only awkward viewing, usually by requiring a switch between move and observe modes; or have provided unrealistic viewing. A common example of unrealistic viewing is "running around looking at the ceiling," which can result from the awkwardness of changing the view direction or having to actively maintain the view characteristics when also devoting attention to complex or rapid movement control (i.e., the inventor perceives this as a case of too much independence in the view controls, for reasons discussed below). In such cases, users can end up disoriented within the simulation and can even become physically nauseous and disoriented in the real world, due to their concentration on the simulation.
A key observation of the inventor is that the above noted requirement to positively and continuously control view is contrary to the way humans usually visually interact with the real world. In day to day human experience, most human vision is automatic, but with the automatic aspects intentionally overrideable. For example, scanning one's view slightly left and right, and slightly up and down are automatic visual activities when driving and walking. In contrast, looking over one's shoulder before changing highway lanes and walking down a corridor and looking left into an open office doorway are common deliberate actions. Notably, in most simulations the above automatic visual actions require deliberate effort and even the above simple deliberate actions are impossible or impractical, due to input system limitations.
Another inconsistency between human interaction with the real world and with simulated virtual environments is hand usage. Humans can, and of course a considerable portion of the time automatically do, perform different tasks with their two hands. Thus, entirely ignoring manipulations (and usage inside the simulation itself), users have two hands which may be used separately for inputting data to perform control functions. Notably, aside from two-handed control of single devices (e.g., keyboards, game pads, and the rare exception like Logitech's WingMan Warrior), two hand control of simulations is today under exploited.
Study, experimentation, and reflection on the above have led the inventor to two novel sets of observations. First, provided the right input system, in scenarios where it is suitable to tie observation to movement, humans can use one hand to control linear movement (e.g., direction and speed) while concurrently using their other hand to control aspects of orientation (e.g., yaw and pitch, or pitch and roll). For reference here, this will be called type one control: extension of user movement capabilities within simulations. Second, again with a suitable input system, humans can use one hand to control movement and concurrently use their other hand to control their view as they move in first person through simulations. This will be termed type two control: addition of user view control to existing movement capabilities within simulations, something which has been awkward or simply not possible in simulations to date. (For purposes of discussion herein, directions of view will be in terms of view pitch and view yaw. The rationale for these designations is that humans typically adjust the non-horizontal component of their view direction by adjusting the pitch of their head or eyes. Similarly, humans adjust the non-vertical component of their view direction by adjusting the yaw of their head or eyes. In contrast, roll type movement is little used for directing human view. Of course, other systems to designate the coordinates needed to describe a direction of view are possible. However, the inventor finds the above form particularly useful in consideration of the dual possible types of control here.)
Type one control is seemingly easily accomplished, after all, joysticks and trackballs have existed for some time and it requires no great imagination to try using one of each concurrently. Unfortunately, for a number of reasons, this simplistic approach has not worked very well.
First, this raises host computer hardware concerns. To use two discrete devices simultaneously necessarily requires having two physical computer ports for their attachment (e.g., serial ports, game ports, or parallel ports). Typically, personal computers do come with multiple ports, but with some already allocated to input devices like mice, printers, modems. Thus, if even possible in a "fully" loaded computer, another input port may have to be added to the host computer. Further, inside the computer, input device address and interrupt conflicts are possible (and are considered by many service personnel, and users who have tried to upgrade their own computers, to be the bane of the personal computer industry). In sum, input systems which require modifying the host computer itself are not generally acceptable in the marketplace.
Second, such an approach unduly burdens simulation creators. Simulations using multiple discreet devices have to accommodate the data and control protocols of the plethora of available devices, as well as of possible future devices. Further, they have to do this by working with the electronic protocols of the computer port hardware available, which may be serial, parallel, game port, custom, or even combinations of these if multiple ports are used. This is a daunting task, which even if it were completely possible, is not at all desirable to simulation creators. From a software (e.g., simulation program) producer's perspective, the preferable approach is to let the producers of hardware (e.g., input devices) handle communications between their devices and the host computer hardware. The preferred means to this end are software drivers. With software drivers the input devices can interface with simulations in a generic manner (e.g., via I/O interrupt, DMA channel, and processor port addressing for data transfer) and giving them data in very generic format (e.g., linear and angular directions, distance, velocity, and acceleration all in standard units). It follows that type one control is preferably accomplished with an integrated approach by the input device maker providing a software driver and only a single available conventional port (e.g., a game port), rather than by merely aggregating existing off-the-shelf input devices. Further, by having the input device maker provide a device driver other benefits are possible. For example, value added features to best exploit the particular input device hardware can be incorporated into the software driver.
All of the above mentioned control systems and the prior art hardware used therein have, however, not satisfied user demand for efficient and comprehensive human first person interaction with computer generated graphical environments. For interacting by moving about, directing one's view, and in some cases performing manipulations within the three-dimensional virtual worlds created by computer simulations users have a present and growing need for improved input control systems and the input equipment upon which such systems can function.